feat: claude models usage, but from bedrock #487
Conversation
|
Thanks! I am interested in supporting bedrock. How are AWS credentials set with this PR? I also noticed a lot of copied code between the bedrock function and the regular Anthropic function. It would be nice to introduce less duplicated code. |
They're supported through AWS's Python SDK (boto3). boto3 accepts credentials in many different ways. On this initial implementation, boto looks for the credentials implicitly on:
We can make that explicit too by passing the credentials directly into the SDK's initialization method. Let me know what works the best for you 🤗
Agreed! Will patch this PR with an abstraction to handle this 🙏 (PS: Done!) |
| CLAUDE_3_5_SONNET_2024_10_22 = "claude-3-5-sonnet-20241022" | ||
| GEMINI_2_0_FLASH_EXP = "gemini-2.0-flash-exp" | ||
| O1_2024_12_17 = "o1-2024-12-17" | ||
| BEDROCK_CLAUDE_3_5_SONNET_2024_06_20 = "anthropic.claude-3-5-sonnet-20240620-v1:0" |
There was a problem hiding this comment.
Not using the latest Claude 3.5 sonnet here (v2, 2024-10-22) because I still don't have access to it on AWS to test it - but should work either way.
@abi we could work on creating a dropdown menu or something on the front-end to allow users to select the model they want to use based on what's available on their environment. Let me know if something like this is already on the radar. It occurs to me that this would be specially useful for Bedrock since equivalent models such as DeepSeek R1 will be available there any time soon.
|
Thanks! I'll take a look soon. |
2 participants
Hey, folks! @abi I've noticed that other people have opened PRs integrating this project with Bedrock (#322, #357 and #367) and I'm opening this PR either way to collect your feedback and whether or not its of your interest to have it merged and be one of the available integrations. Let me know! I'm willing to treat this PR carefully to match the expected standards.
Amazing project by the way. Great work putting this up 🙏